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ABSTRACT 

A study examined school personnel perspectives on 
barriers to the implementation of a statewide program to influence 
change in curriculum and instruction. Interviews were conducted with 
personnel from 5 schools and districts and generally positive results 
were reported in moving from their existing curriculum and assessment 
programs to the mandated state program. Interview data from teachers, 
principals, and curriculum coordinators demonstrated that adherence 
to the mandated statewide program was not without considerable 
challenges. School personnel reported that the implementation of the 
program and intended school change was made difficult by barriers 
related to the lack of alignment between existing and mandated 
instruction and performance assessment, lack of alignment between 
teacher practices and beliefs with those implicit in the statewide 
program, lack of resources to help implement change mandated by the 
state program, the performance assessment materials and procedures 
themselves, and the lack of sufficient communication from the 
state-related to the mandated program. Results suggest that 
overcoming the barriers to implementation of the statewide program 
requires a systematic approach that provides the resources to support 
change of classroom practice, helpful communicat ion between those 
people involved in the curriculum, instruction, and asses sment change 
processes, and ongoing refinement of the assessment program and the 
performance assessment materials and procedures. (Contains 21 
references.) (Author/RS) 
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Abstract. This study examined school personnel 
perspectives on barriers to the implementation of a 
statewide program to influence change in curriculum 
and instruction. We interviewed personnel from five 
schools and districts that reported generally positive 
results in moving from their existing curriculum atxd 
assessment programs to the mandated state pro- 
gram. Interview data from teachers, principals, and 
curriculum coordinators demonstrated that adher- 
ence to the mandated statewide program was not 
without considerable challenges. School personnel 
reported that the implementation of the program and 
intended school change was made difficult by 
barriers related to the lack of alignment between 
existing and mandated instruction and performance 
assessment, lack of alignment between teacher 
practices and beliefs with those implicit in the 
statewide program, lack of resources to help imple- 
ment change mandated by the state program, the 
performance assessment materials and procedures 



themselves, and the lack of sufficient communication 
from the state related to the mandated program. The 
results suggest that overcoming the barriers to 
implementation of the statewide program requires a 
systemic approach that provides the resources to 
support change of classroom practice, helpful 
communication between those people involved in the 
curriculum, instruction, and assessment change 
processes, and ongoing refinement of the assessment 
program and the performance assessment materials 
and procedures. 

Large-scale reading assessment is increas- 
ing in frequency and changing in form and 
purpose. There is a burgeoning use of large- 
scale assessments to measure student achieve- 
ment and school effectiveness at state and 
school district levels; a survey determined that 
38 states assess the reading abilities of students 
in at least one grade level (Afflerbach, 1990). 
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These large-scale assessments yield information 
that is used by different audiences for a variety 
of purposes. Taxpayers, elected officials, 
parents, school board members, teachers, and 
others regularly use the results of large-scale 
state assessments to determine the achievement 
of students, the effectiveness of reading 
instruction programs, and the accountability of 
schools and teachers. Large-scale assessment 
also has been used to intentionally influence the 
nature of schooling: what is taught and how. it 
is taught, and what students learn. 

Recent development and implementation of 
assessment have been influenced by a dissat- 
isfaction with traditional forms of student 
assessment and the promise of more authentic 
and useful assessment materials and procedures 
(Shepard, 1991; Valencia, Hiebert, & Affler- 
bach, 1994). For example, large-scale assess- 
ments of reading have often been conducted 
using multiple-choice tests that require students 
to read short passages for which they have little 
prior knowledge, and to choose one correct 
answer from among distractors. The resulting 
student responses to items present a severely 
restricted assessment of students’ reading 
achievement. Such tests have questionable 
ecological validity when they lack a relation to 
the teaching and tasks that help readers learn to 
read, and to how students understand and use 
what they read in the classroom. Recent inno- 
vations, such as performance assessments, 
allow students to read complete texts (or sever- 
al texts) and to construct responses to the texts 
they read. These constructed responses may 
allow students to demonstrate general compre- 
hension of text, as well as individual response 
to the text. Furthermore, performance assess- 



ments embed acts of reading in contexts that 
encourage or require students to perform tasks 
using the information gained from reading. 

Concurrent with the evolution of large- 
scale assessment is the evolution of classroom 
curriculum and instruction. Repeated assertions 
that students in the United States are not 
achieving to their potential (Mullis, Campbell, 
& Farstrup, 1993; National Commission on 
Excellence in Education, 1983; Williams, 
Reese, Campbell, Mazzeo, & Phillips, 1995) 
have led to the development of curriculum and 
teaching that encourages students to perform 
tasks that include the synthesis of information 
gathered through the reading of text, complex 
problem-solving tasks, and collaboration with 
classmates. Such teaching and learning should 
help students move beyond basic literacy skills 
to more involved reading and reading-related 
performances that demonstrate the value and 
the uses of reading in the real world. 

The dual initiatives of developing better 
assessments and influencing instructional 
change have been melded in programs such 
as the Maryland School Performance Assess- 
ment Program (MSPAP). In these high-stakes 
assessments, tasks, materials, and learning 
outcomes are presumed to be exemplars of 
what students should do, use, and learn. An 
underlying assumption here is that quality, 
large-scale assessments can be a positive influ- 
ence on school achievement. However, the 
concerns related to large-scale assessment 
programs are considerable. The feasibility of 
large-scale, high-stakes performance assess- 
ment should be examined from both a measure- 
ment perspective (Linn, Baker, & Dunbar, 
1991; Schafer, Guthrie, Almasi, & Afflerbach, 
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1994), and from school budgeting and schedul- 
ing perspectives (Shavelson, Baxter, & Pine, 
1992). Given the concerns with the reliability 
and validity of performance assessments, the 
investment of resources of time and money 
must be carefully considered. 

There are further concerns about possible 
negative impacts of large-scale, high-stakes 
assessment. The success of an assessment 
program depends on more than the psycho- 
metric rigor of the instruments and procedures 
used. Useful assessments are informed by a 
systemic perspective (Frederiksen, 1984; Smith 
& O’Day, 1991) that can contribute to the clear 
communication of the nature and intent of the 
assessment to those who take and use the 
assessment, and to the coordination of assess- 
ment efforts to efficiently yield useful infor- 
mation (Afflerbach, 1995). When assessment 
drives instruction, it may limit students’ 
classroom experiences to materials and tasks 
that are congruent with the assessment (Smith, 
1991). Teachers, parents, students, and princi- 
pals may have quite different ideas about what 
“effective” instructional practice and materials 
are, and what teaching and assessment are most 
valid and appropriate for their own classrooms 
(Tittle, 1989). However, the top-down nature 
of large-scale assessments (i.e., they are man- 
dated, developed, and overseen by states or 
districts) and the curricular reform they are tied 
to often assures that a particular definition or 
conceptualization of teaching and learning will 
prevail. This can cause considerable conflict 
among teachers who are convinced that they 
are already providing effecti/e instruction for 
their students. Mandated large-scale assess- 
ments may also result in the diminished profes- 



sionalism of teachers and the alienation of 
teachers if (1) the assessment is considered the 
single indicator of school and student success 
(Johnston, Afflerbach, & Weiss, 1993; Smith 
& Rottenberg, 1991), or (2) there are consider- 
able differences between individual teachers’ 
and states’ model of students’ literacy develop- 
ment, and how best to measure it. In sum, 
mandated curriculum, instruction, and assess- 
ment programs exist in social and political 
contexts. Each may influence the other and it 
is reasonable to expect that massive change 
may encounter barriers to that change. We 
planned the study reported here accordingly. 

The investigation of barriers to the 
implementation of a large-scale performance 
program as reported by school personnel 
may help describe the appropriateness and 
effectiveness of ongoing and future efforts to 
influence school change through large-scale 
assessment. It may also infonn ongoing and 
future efforts to maximize any positive influ- 
ences of large-scale, high-stakes performance 
assessments, and contribute to the development 
and construction of performance assessments 
themselves. Finally, the examination of barri- 
ers helps describe the consequential validity of 
this statewide performance assessment in the 
broadest sense: what happens to teachers and 
students as a result of an assessment, or as a 
result of an assessment program? 

The Maryland School Performance Program 

This study focused on barriers to the 
implementation of a large-scale, statewide 
performance program in reading, as reported 
by the school personnel who administer, use. 
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and are influenced by the assessment. The 
Maryland School Performance Program was 
mandated by the state legislature in an effort to 
help all students learn and achieve in their 
respective schools. Tne Maryland School 
Performance Program is founded on three 
fundamental premises; 

• all children can learn 

• all children have the right to attend schools in 
which they can progress and leam 

• all children shall have a real opportunity to 
leam equally rigorous content 

(Maryland School Performance 
Program Office, 1990, p. iii) 

The school perfomriance program serves several 
purposes that are regularly associated with 
high-stakes assessment; a means of driving 
instructional reform, determining school 
luality, and proving school accountability. In 
addition, it includes a series of instruments that 
can influence instruction in a positive (from the 
perspective of supporters) manner and it is also 
a tool for measuring school accountability. 

. . .the Maryland School Performance 
Program has developed a comprehensive 
accountability system based on results and 
incorporating excellence and equity for 
each student. (Maryland State Perform- 
ance Program Office, 1990; p. iii) 

The school performance program is a high 
stakes assessment. Information from the school 
performance program is used by the state to 



decide if schools are helping students achieve 
particular levels of academic performance. 
School-level reading scores are reported as part 
of a school profile which includes performance 
on reading and math assessments, and school 
attendance and dropout rates. Schools whose 
students do not meet the state guidelines for 
achievement are at-risk for takeover by the 
state, which can be realized by the replacement 
of the teaching and administrative staffs of 
entire schools. In addition to providing ac- 
countability information about schools and 
school districts, results from the school perfor- 
mance program are eventually reported back at 
the school and district levels to provide 
feedback about how well they are teaching 
children. Results are not reported at the 
individual student level. 

The reading portion of the school perfor- 
mance program examines student achievement 
in relation to the outcomes of; (1) positive 
attitudes toward reading; (2) construction, 
extension, and examination of meaning when 
reading for literary experience; (3) construc- 
tion, extension, and examination of meaning 
when reading for infomiation; (4) construction, 
extension, and examination of meaning when 
reading to perform a task; and (5) demonstrat- 
ing awareness of strategic behaviors and knowl- 
edge about reading (Maryland State Performance 
Program Office, 1990). Tasks on the perform- 
ance assessment are open-ended (i.e., there is 
no single predetermined correct answer), and 
tasks require students to engage in various 
literate activities including reading and re- 
sponding to text in collaboration with peers, 
reading several texts related to one topic, and 
writing in response to reading. The assessment 
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considered in this study was fully intended to 
shape instruction and learning, and perfor- 
mance assessment tasks and materials were 
intended as models for instruction. In taking 
the performance assessment, students were 
asked to perform tasks that the state education 
department considered important to the literacy 
development of third- and fifth-grade students 
(Kapinus, Collier, & Kruglanski, 1994). 

Methods 

Schools and participants 

Personnel from five schools in five differ- 
ent districts participated in this study. The 
schools were selected through a process that 
began with our establishing contact with 21 of 
24 school districts in the State of Maryland. 
Initial requests were for school district person- 
nel to participate in an interview to identify 
schools that were implementing change as a 
result of the school performance program. The 
district representatives included supervisors 
of reading (5), supervisors of elementary 
education (5), supervisors of instruction (3), 
supervisors of language arts (3), directors of 
curriculum (2), a reading specialist, a princi- 
pal, and a district coordinator of testing and 
Chapter One. The district representatives 
received letters stating the purpose of the 
research project, and the questions that would 
be asked during the interview. Next, telephone 
interviews were held with each district represen- 
tative. The interview consisted of 13 questions 
that were developed to help us: (1) identify 
schools within each district that were initiating 
changes and innovations in response to the 



school performance program; and (2) deter- 
mine what kinds of changes were occurring at 
the county level in response to the school 
performance program. The 21 district repre- 
sentatives nominated 42 elementary schools 
that they believed were demonstrating positive 
change in relation to the school performance 
program, and change in the direction supported 
by school performance program. From this 
group we chose schools from 5 different 
districts. 

Procedures 

Data was gathered at each of the five 
schools with an 8-item, semi-structured inter- 
view, which was developed by the authors. 
Many of the interview questions were similar 
to those asked of district representatives 
during the “school finding” phase of our 
work. We piloted each question with teachers 
and administrators who worked in Maryland 
public schools, but who did not participate in 
the study. Information from the piloting con- 
tributed to final revision of the interview 
question set. The first 7 questions centered on 
participants’ perceptions of the state goals, the 
impact of the school performance program on 
their schools and classrooms, and the manner 
in which change and innovation had taken (or 
was taking) place. The final interview question 
asked participants: 

What barriers (if any) did yea face trying 

to make these changes? 

This report focuses solely on the issues of 
barriers to school change in curriculum. 
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instruction, and assessment practice related to 
the Maryland School Performance Program. The 
data for this study were collected at the end of 
the second year of implementation of the pro- 
gram. More complete consideration of the 
impact of the program on classroom curriculum 
and instruction as indicated in response to 
other interview questions is contained in 
related reports (Almasi, Afflerbach, Guthrie, 
& Schafer, 1994; Guthrie, Sciiafer, Afflerbach, 
& Almasi, 1994; Schafer et al., 1994). 

Audiotaped interviews and field notes were 
collected in 1-day or 2-day sessions in each 
school district over a 3-month period. All 
interviews were arranged by a building 
administrator prior to the visits for data collec- 
tion. Interviews were conducted with building 
administrators (principals), curriculum special- 
ists (curriculum coordinators, reading/language 
arts supervisors), and teachers (reading special- 
ists, third- and fifth-grade teachers). Interviews 
were conducted in schools and district offices. 
Most interviews were held with individuals. 
However, teachers were interviewed in groups 
to accommodate their teaching schedules. All 
participants were selected because of their 
experiences and knowledge regarding the 
Maryland School Performance Program, and 
their diverse knowledge regarding curricular 
and instructional change within their particular 
school. Transcripts of the taped interviews 
were the data source. 

Data Analysis 

The first phase of data analysis involved 
two of the researchers reading all responses to 
the barriers question independently and identi- 



fying barriers described by the various partici- 
pants in the study. Next, the researchers met to 
disc: ss their individual interpretations of the 
data, to develop classes of barriers as they 
were grounded in the participants’ interview 
data, and to discuss specific instances of barri- 
ers (Glaser & Strauss, 1967). Each instance of 
a barrier described by study participants in the 
interviews was recorded on a master list, and 
the researchers generated a set of general 
categories that could explain the relationships 
of particular barriers, while efficiently group- 
ing the growing list of barriers. The processes 
of identifying, discussing, recording, grouping, 
and regrouping potential barriers resulted in 
the increased conceptual density of each initial 
category. This resulted in the formation of five 
categories of barrier to the implementation of 
Maryland School Performance Program: (1) lack 
of alignment of current and mandated instruc- 
tion and assessment; (2) lack of alignment 
between teacher practices and beliefs and 
those underlying the mandated assessment and 
instruction; (3) lack of resources needed to 
meet the demands placed on schools, teachers, 
and students by the new performance program; 
(4) the nature of the performance assessment 
itself, including administration and reporting of 
results; and (5) the quality of communication 
related to the nature and intent of the perfor- 
mance program and assessment. 

Results and Discussion 

This section features a representative 
sample of participants’ responses to the request 
to describe barriers to the implementation of the 
school performance program. We have selected 
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interview excerpts that we feel best illustrate 
individual school personnel perspectives on 
the five types of barrier. Each interview 
excerpt is identified by a label that signifies 
the school and participant. Several aspects of 
our data gathering influenced our decision to 
describe classes of barriers and to forego the 
examination of barriers in the contexts of 
individual school’s structures, programs, and 
communities. The interview sessions were con- 
ducted as school personnel were available, and 
this resulted in single interviews with some 
participants and group interviews with other 
participants (i.e., classroom teachers). A result 
is that the interview contexts, formats, and 
numbers of participants were not equivalent 
and it was not dear to us if individually stated 
practices and beliefs were generalizable to 
the groups of teachers we interviewed. We 
present participants’ descriptions of particular 
barriers while noting that some of the interview 
excerpts include references to more than one 
barrier. For example, one excerpt describes a 
lack of school funds that might help teachers 
work to align the school curriculum with the 
performance program. This highlights an 
overlap between barriers related to resources 
and alignment. In such cases, we classified the 
participant’s response in terms of the most 
prominent barrier mentioned. 

Lack of Alignment of Current and Mandated 
Instruction and Assessment 

The Maryland School Performance Program 
is intended to change and support particular 
teaching practices. An initial barrier for some 
teachers was the change from their current 



reading instruction to instruction aligned with 
the new statewide performance reading assess- 
ment. A lack of alignment between her current 
classroom practice and that assessm.ent was 
noted by a fifth-grade teacher. This teacher 
had an instructior>aI goal of helping students 
develop independence in performing classroom 
tasks, while the statewide performance assess- 
ment encouraged helping students to demon- 
strate achievement through collaborative work 
with classmates: 

They (the students) really don’t like work- 
ing independently. They would rather 
work in a group. Wouldn’t most of us? 

Like we are doing right now where we 
bounce ideas off here and there, instead of 
just sitting here and answering questions 
by yourself. Yet we feel that somewhere 
else down the line they also need to be 
independent thinkers on their own. 
(RF.0402) 

Convinced that her goal of helping students 
develop independence was important, the 
teacher had designed significant amounts of 
classroom instruction and had altered the 
classroom environment to meet this goal. Yet, 
she felt she was required to design collabora- 
tive tasks to best prepare her students for 
taking the performance assessment. 

The school performance program was 
intended to change classroom teaching and 
learning, and it resulted in differences in stu- 
dent learning outcomes. With the changes in 
nature of student outcomes came the need for 
reporting changes in student achievement that 
were aligned with new curriculum. Most 
districts used report cards that reflected their 
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particular curriculum (and consensus values) to 
report student achievement and progress to 
parents, students, and other teachers. The 
goodness of fit of the categories and descrip- 
tions within the report card with what students 
learned as a result of the Maryland School 
Performance Program became a barrier. There 
was a lack of alignment between the students’ 
learning outcomes from instruction developed 
as a result of the school performance program, 
and the means of describing the outcomes using 
the report card. This meant the need for the 
development of new report cards: an effort that 
demanded considerable time of teachers and 
supervisors, in addition to the curricular 
changes mandated by school performance 
program. A fifth-grade teacher told us: 



First grade has developed a report card for 
their grade level, but we’re still using the 
original book report cards. So that could 
be something that they could look into . . . 
helping create a report card for this type 
of reading and writing strategy. (RT.0401) 



The Maryland School Performance Pro- 
gram was considered an innovative assessment 
by some teachers and a flawed assessment by 
others. Regardless of participants’ beliefs about 
the program, it was mandated by the state and 
it was used to measure school goodness. In 
effect, it required that all teachers adjust their 
instruction and curriculum. The lack of align- 
ment between current classroom instruction and 
the instruction that prepared students to take 
the school performance program influenced 
teacher confidence. This resulted in consider- 
able anxiety for some teachers, anxiety that 
itself could be a barrier. Teachers’ work to 



align instruction occurred in a context in which 
teachers were under considerable scrutiny and 
accountability. Teachers reported that they had 
varied levels of confidence in teaching with 
new materials and new instructional proce- 
dures. A third-grade teacher told us: 



Literature-based instruction is harder tlian 
teaching from basals. It is harder. It is 
much easier to pick up a teacher’s manual 
and follow. . . it’s much easier . . . and we 
feel ... I know I still don’t feel confident 
. . . that you are covering everything that 
you’re supposed to cover and doing every- 
thing that you’re supposed to do. (RF.0401) 



In addition, teachers were concerned about 
the alignment of existing classroom assess- 
ments with the new mandated curriculum, and 
their ability to collect diagnostic or formative 
assessments of student development in a school 
performance program-informed instructional 
program. A third-grade teacher reported: 



You also have a little bit of insecurity with 
the littler guys still going back to that 
question, “Where are they?” You still 
kind of like that security. Are they on the 
first grade? Are they on preprimer? 1 think 
that’s why we bought so heavily into the 
portfolio assessments because they were a 
little bit more security for us because we 
could look and see how they were devel- 
oping and how they were moving along 
with their level of security (M.0501) 



The school performance program includes 
the large-scale performance assessment system 
that was developed outside the culture and 
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practices of schools and classrooms, with the 
intention of changing school practice. While 
all participants in this study understood the 
mandate for change, many were not sure of the 
details of change. Similarly, teachers were not 
sure who or where they might consult to deter- 
mine if the changes they implemented in their 
classrooms were appropriate. Lack of famili- 
arity with the specifics of change necessitated 
by the school performance program and lack of 
clarity of implications for instruction were a 
barrier for this reading teacher: 

Sometimes tying it all together is difficult 
because we were talking and we tried to 
pull this into here and this into here and 
sometimes it just doesn’t quite fit there at 
that time. They may fit somewhere else. 

So our thing’s been basically going 
through these novels to see where they 
would fit. Maybe next year we’ll be better 
because we’ll know if this fits better here 
as opposed to putting them with this unit. 
(M.0503) 

Lack of Alignment of Mandated Instruction and 
Assessment with Teacher Practice and Belief 

Large-scale assessment can have varied 
influence on teachers and their teaching. Nega- 
tive effects of high-stakes assessment include 
the alienation that some teachers feel when the 
assessment reduces teachers’ initiative to make 
classroom decisions, and a lessened sense of 
professionalism that occurs when teachers are 
required to teach what the state education 
department decides is important (Johnston et 
al., 1993; Smith, 1991). The teachers partici- 
pating in this study did not cite alienation or a 



sense of reduced professionalism as barriers to 
implementing the school performance program . 
Still, we gathered information on the crucial 
nature of teachers’ willful participation in the 
changing of their practice, as described by a 
reading supervisor, building principal, and a 
teacher. The following three excerpts describe 
different participants’ perspectives on the 
degree of alignment between individual 
teacher’s classroom practice and the practice 
required by the school performance program, 
and teachers’ willingness to consider change in 
their practice to align with state goals and 
objectives. 

Teachers and supervisors who had confi- 
dence in their reading instruction and reading 
program might resist the change mandated by 
the school performance program, believing that 
there is no need to change their instruction. 
Resistance also might have roots in the alien- 
ation felt by teachers who consider themselves 
professionals but who feel they are not treated 
professionally in a program with mandated (as 
opposed to suggested) change. A third-grade 
teacher told us that success in the new program 
and on the performance assessment might be 
highest with teachers who were already achiev- 
ing in their classrooms. In contrast, the teacher 
told us that teachers determined to set their 
own curricular agendas might have difficulty. 

The teacher is the crucial barrier. You can 

do all the training you want and if the 

teacher has a closed mind and won’t try 

it— you can forget it. (CW.0201) 

This teacher did not report having any personal 
dilemmas related to teaching in terms of the 
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school performance program, but acknowl- 
edged the key role of the teacher to carry out 
the externally mandated plan. 

A reading supervisor described teachers’ 
and students’ use of a writing process portfolio 
procedure that was directly informed by the 
performance assessment. The specific demands 
of the portfolio assessment processes and 
contents forced teachers toward a particular 
type of writing instruction which was aligned 
with the writing required on the performance 
assessment. Portfolios were used for the pur- 
pose of proving school and teacher accountabil- 
ity to the state program, as well as for their 
more familiar purpose of documenting and 
facilitating student growth. 



The kids work out of these folders during 
the writing process and this was kind of a 
check and balance on the teachers to make 
sure they were doing the whole writing 
process. (CC.0304) 



A building principal reported special 
concerns with recently graduated and newly 
certified teachers. He saw his role as one of 
inculcating new teachers and creating conform- 
ity in their practice in relation to the school 
performance program. Alignment of teaching 
practice with school performance program 
guidelines and expectations was paramount for 
this principal, even if teachers had practices 
and beliefs that differed from those compatible 
with the state program 



Our new teachers are my worry. We have 
a few new teachers and I can’t get them 
where I want them to be, but we’re work- 
ing on it. (CC.0305) 



Lack of Resources 

A frequent barrier to implementation of 
changes in curricular practices and materials 
was the lack of district or school resources. 
Participants most often cited lack of resources 
in terms of time and money, with time the 
most frequently cited barrier to implement- 
ing school changes related to the school 
performance program. Many of the partici- 
pants in this study noted that the new assess- 
ment required substantial time commitments 
to perform an array of tasks. The school 
performance program demanded teacher 
responsibility (and related demands of time) 
for planning and monitoring instruction, chang- 
ing instruction, and observing and evaluating 
students. The school performance program was 
designed to encourage changes in teaching and 
learning that are fully aligned with the perfor- 
mance assessment, but the process of change 
and the lack of time needed to change defined 
a barrier, as reported by a reading supervisor. 

Time. Not enough time. There’s not 
enough teacher planning time, and I just 
don’t know how to get around it. ... I 
would say that that’s been the biggest 
obstacle. . . , getting enough time to sit 
down and realty collaborate the plans and 
work together. (CC.0304) 

Participants reported that the school perfor- 
mance program resulted in changes in their 
lives both inside and outside of school. The 
interview data indicated that most teachers 
were willing to give the extra time demanded 
by the performance program to effect change. 
Yet, the teachers acknowledged that too heavy 
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a demand on their time created a considerable 
barrier, despite their commitment to changing 
their own practice and student learning. A 
second reading supervisor reported: 



It’s a hard place to work in. We don’t get 
out of here until 6:30 at night lots of times 
and when I leave there’s still cars from 
teachers out there. I come in on Sundays 
sometimes. (RT.0104) 



their instruction and to share their approaches 
to changing instruction to align with the school 
performance program and the new assessment. 
However, some schools could not provide the 
time to allow teachers to do so, as reported by 
a fifth-grade teacher; 



We need the time to sit down together as 
a team and try to put it together . . . and 
we don’t have the time. (RF.0402) 



Even in school districts that provided support 
for implementing change (usually in the form 
of salary for work beyond contract hours or 
added inservice days), the time allotment was 
not always sufficient. A third-grade teacher 
reported: 



They [the county Board of Education] did 
give us inservice time, but a lot of it [the 
amount of work to prepare lessons] is 
extra nights and extra hours, tmly. In 
these folders [their lesson plans for each 
trade book] we try to approach the dialog 
books [journals], or the trade books, 
through the state’s stances— the global, 
interpretive, personal, and critical— so that 
when we do our com,prehension activities, 
we are trying to cover those that are used 
in the assessments with the school perfor- 
mance program, but I don’t think any of 
us looked at it as, “Oh, this is getting the 
students ready . ’’ (RF .040 1 ) 



Several teachers told us that their preferred 
approach to planning and implementing change 
at the third- and fifth-grade (the grades at 
which the MSPAP was administered) level was 
to meet regularly in collaborative work gr<oups. 
These meetings allowed teachers to integrate 



In response to the demands of the school 
performance program, some districts allotted 
funding for additional teacher work time and 
inservice sessions that were intended to help 
teachers move from their current classroom 
practice to that practice suggested by the school 
performance program. A third-grade teacher 
noted that the school district allocated time for 
inservice training to help teachers encourage 
students’ written responses to what they read. 
This teacher determined that inservice training 
was inconsistent. This teacher’s colleagues did 
not necessarily receive the same training. 



The county has also provided some ideas 
for response to writing through the ILA 
[Integrated Language Arts] committee. 
The only trouble is that the way that the 
inservice [is divided] up between math one 
year and ILA the next . . . because there 
are two focuses in the county, not every- 
one’s inservice is the same amount of time 
or on the same topic. (CC.0401) 



Closely related to investments of time were 
district and school financial resources. Changes 
in instruction required new materials and 
training for teachers. The state mandated and 
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delivered the school performance program 
goals and performance assessment, and the 
directive to align teaching and learning with the 
school performance program. Yet, individual 
districts and schools had to provide resources 
to effect this change. One typical resource was 
inservice work, but the nature and effectiveness 
of inservice work was influenced by school 
and district budgets. A reading supervisor 
described the different budget demands that 
competed with the need to help orient teachers 
to the new program of performance assessment 
and related curriculum; 



We’ve got the state outcomes to meet and 
the changing curriculum, and we have 
been cut back on inservice time. I mean, 
we used to have substitutes and at least 
time for teachers to go to training. And 
that has all been cut because of the budget. 
(CC.0304) 



The literacy portion of the performance assess- 
ment required students to read and use their 
understanding of what was read to perform 
academic tasks. The tasks included respond'^® 
in writing after gathering information front a 
variety of reading materials and reading to 
comprehend information in relation to other 
forms of communication. While these assessment 
tasks have the potential for high ecological 
validity (in classrooms, students may gather 
information to perform tasks from an array of 
texts, from discussions with classmates, and 
through observing movies and television), 
instruction and assessment that incorporates 
diverse forms of media requires specific 
resources. The importance of school resources 



required by the new school performance pro- 
gram and assessment was emphasized by a 
third-grade teacher; 



We started without a media center that 
was functional because they didn’t fund it. 
(RF.0401) 



Money pays for instructional materials and 
training related to the mandated performance 
program. Not all teachers were satisfied with 
the level of funding provided at the building or 
district level, and the frustration this caused 
was evident. A third-grade teacher reported the 
barriers related to lack of classroom materials 
and the time to develop appropriate instruction; 



The materials and the time. It all comes 
down to money because really they want 
us to make all these changes and do these 
neat things and for elementary teachers 
that’s a big thing. (RF.0401) 



In addition, some teachers felt the need to limit 
how much of a personal and professional 
commitment they would make to a program 
that was mandated by the state, but not funded 
by the state. A fifth-grade teacher in the same 
school told us; 



They say we get a lot of inservice, but 
there isn’t a lot of inservice. We got 
inserviced before the school year when 
we’re all tense about getting our classes 
set up. You are asked to stay after school 
or work on Saturdays, but there is no 
money to pay to work during the summer, 
which is the ideal time. (RF.0402) 
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Performance Assessment Materials and 
Procedures 

Participants reported barriers related to the 
use of performance assessment materials and 
procedures themselves. These included the 
difficult nature of assessment tasks for some 
students, the novelty of the assessment format, 
tasks, and the classroom environment in which 
the assessment takes place, the administration 
of the assessment, and the scoring and report- 
ing of the school performance program. The 
assessment itself may become a barrier if it is 
too difficult (students may become discour- 
aged, frustrated, or embarrassed), if it requires 
too much time from the instructional year of 
the teacher and students, or if it is perceived by 
the teacher as unfair to particular students. 

Teachers’ greatest concern about the per- 
formance assessment itself was the level of 
difficulty of student tasks. One teacher reported 
her concern thai .he assessment placed some 
students in a situation in which failure was 
assured. A feature of the school performance 
program is that all students at grades 3, 5, and 
8 are required to take the assessment, regard- 
less of their reading ability. The teacher’s 
knowledge that a particular child may not be 
able to perform or complete the assessment is 
not considered. A result is that the students 
who are known to be nonreaders (or readers 
performing well below grade level) may suffer 
embarrassment from a lengthy and public 
display of failure. This translates into a barrier 
when it leads to teachers’ skepticism with the 
assessment, and teachers’ beliefs that the poten- 
tially negative effects of the assessment 
experience are not worth the information it 



may yield. A fifth-grade teacher told us of an 
experience with a low-achieving student: 

When we’re working with kids who are 
really low and slow ... I mean you can 
tell ... we can tell . . . “This kid’s going 
to be a ‘5’” and there’s nothing we can do 
about it ... and there they are sitting 
there trying to take the test. (RT.0102) 

For students, it may lead to degradation of 
already low self-esteem or self-confidence. 
Here, the teacher may feel the need to protect 
their students from public failure, or a testing 
experience that they believe could be devas- 
tating to their students. Alternatively, the 
above response could be interpreted as a 
teacher having low expectations for a particular 
student. In either case, there was no option 
for the teacher to recommend student exemp- 
tion from the assessment. This may lead to 
teacher alienation, or student reluctance to 
read further. 

Participants raised several concerns about 
the administration of the performance assess- 
ment. In particular, the testing sessions were 
considered too lengthy, and eventually exhaust- 
ing for students and teachers. This was reported 
by this fifth-grade teacher: 

My major concern about the test is that I 
think it should be broken down more 
instead of just hitting them in one week. 
(M.0502) 

In addition, the amount of class time (one 
week) dedicated to administering the perfor- 
mance assessment seemed extreme to some 
teachers. A reading teacher reported: 
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The first day of the school performance 
program is fine, but it seems like when we 
get to the end of the cycle it’s just too 
much. Even if they could give the test with 
one part earlier in the year and then give 
another part some other time in the year or 
something so that they (students) don’t get 
it all at one point. (CW.0203) 



A consistent set of concerns was voiced 
about the administration of the performance 
assessment. Some performance and collabora- 
tive tasks appeared to have been designed by 
persons who had little knowledge of the ele- 
mentary classroom, or of the feasibility of 
performing particular tasks in classrooms. For 
example, test security procedures required that 
teachers did not receive any part of the assess- 
ment prior to the day of the assessment. One 
performance task required that students mix 
compounds on their desks and this created 
considerable confusion because teachers were 
not allowed to prepare classrooms before 
administering the test. The resulting barrier 
here was related to teacher faith in the assess- 
ment, and skepticism that whoever designed the 
assessment did not understand students or 
classrooms. A fifth-grade teacher reported that 
test security and standardization measures 
created major problems when students were to 
perform tasks that involved dirt and clay. 



I was wondering what fool made these 
things [particular assessment tasks] up . . . 
and then you’re not supposed to deviate at 
all from the instructions. There was no 
instruction to cover the desks with news- 
paper! (RT.0102) 



An additional concern was voiced over the 
creation of novel and unfamiliar groups of 
students to take the school performance pro- 
gram. It is performance assessment policy to 
divide intact classrooms of students and admin- 
ister the performance assessment to groups of 
students from several classes. Students often 
are not in their regular classroom, and they 
may be placed with unfamiliar students. Thus, 
students take the assessment in an unfamiliar 
classroom setting. The resulting student anxiety 
may be a barrier to student achievement on tiie 
performance assessment, to the degree that the 
testing context differs from the routine of the 
regular classroom. A reading specialist told us: 



Another problem is the intermixing of kids 
in the groups, for example ... my kids 
[the gifted class] are used to working with 
each other and they know how to work 
with each other in the classroom .... 
During the testing situation you have kids 
that come from other classrooms that are 
not quite as on target .... They’re not as 
focused and it’s like they don’t know what 
to do (CW.0203) 



Participants also had concerns with the 
scoring procedures used for the performance 
assessment. One teacher noted that student 
writing was confounded with reading achieve- 
ment and that the score yielded by this con- 
founded measure might not be a valid, accurate 
reflection of student learning or the teacher’s 
ability. In addition, the confounding of reading 
and writing caused skepticism about the assess- 
ment among school personnel who were familiar 
with basic concepts of educational measure- 
ment. Results from an assessment that describe 
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children who have not learned to read (and by 
implication, a teacher who has not been suc- 
cessful teaching them) may cause considerable 
barriers to further implementation of the pro- 
gram, especially when the results have low 
construct validity (i.e., not clear what is 
contributing to a student’s performance) and 
potentially great consequential validity (i.e., 
teachers and students are judged by assessment 
results that have high stakes). A fifth-grade 
teacher described poor handwriting, student 
anxiety, and language difficulties as possible 
influences on students’ performance. 



I think that kids who have illegible hand- 
writing shouldn’t be lumped in with the 
kids who don’t do . . . who can’t do it [the 
assessment task]. . . . Let’s not lump them 
all together because if you’ve got so many 
kids in category “5,” does that mean our 
school is doing an awful job? [With] some 
of our kids, it’s because they can’t speak 
English . . . [with] some it’s because they 
are scared. (RT.0102) 



The amounts of school, classroom, and 
individual teacher and student time and effort 
that were invested in teaching to, preparing 
for, and administering and taking the school 
performance program were considerable. In 
contrast, the return on investment was slight 
and distal. State performance assessment re- 
sults were not reported at the individual student 
level, and reporting back from the state to 
districts and schools took in excess of a 
year. A result is that the high-stakes assess- 
ment provides no information useful to the 
classroom and that participation in the perfor- 
mance assessment lacks meaningful closure for 



teachers and students. A fifth -grade teacher 
told us of her concern that the considerable 
time invested in preparing for and administer- 
ing the school performance program yielded no 
useful information. 

You would think that there would be an 
analysis of the kinds of errors and why 
that student is making that kind of error. 
(RT.0102) 

Students received no diagnostic or personally 
useful information about their performance, 
and teachers received no information that 
might be used to reflect on and revise or 
maintain their instruction. A district reading 
supervisor told us: 

The downside of performance assessment 
of course, is that we don’t get responses 
back. The way the data is reported back to 
us is on a scale of 1 to 5 and various 
rubrics that we use to grade. If you get a 
5, you figure you didn’t do it right and if 
you got a 1 , you figure you did really . . . 
scored well. If you came in with a 2 and 
80% of them [the students] did it, you 
must be pretty good on those rubrics. 1 
mean you’re making a lot of guesses. 
(RT.0106) 

Communication Related to the School Perfor- 
mance Program and Assessment 

A final set of barriers was related to the 
nature and frequency of communications 
between the state education department that 
developed the assessment, and the districts and 
school members who had to administer, take. 
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and interpret the assessment. Clear communi- 
cation is imperative when high-stakes assess- 
ment and instructional change initiatives are 
developed by a state education department and 
mandated for use in all schools and classrooms 
in the state. Ccnununication is clearly connected 
to many of the barriers cited throughout this 
paper. For example, characteristics of the test 
might be better understood with clear commu- 
nication. and clear communication might help 
the school district maximize the resources of 
time and money for training. Furthermore, 
successful communication might help school 
personnel better understand the nature and 
intent of the performance assessment, and 
this might help convince school personnel of 
the positive aspects of school performance 
program. 

An example of the first type of communi- 
cation barrier is related to contradictory 
messages that a district supervisor received 
from the state education department. The 
communication was unclear, and it demonstrated 
a lack of understanding on an administrative 
level that created barriers to implementation at 
the classroom level. 

They [the state] tells us not to use the 
results for instructional purposes and 
then they have all the elementary school 
principals in school this summer and 
started telling us, “You better think 
seriously about using what data we have 
for instructional purposes.” They gave us 
suggestions— “Dimensions of Learning”— 
and how t6 apply [them] and more things 
like getting teachers to know rubrics and 
understand what rubric is in terms of 
assessment. (RT.0106) 



Conumumcation between school and home is 
crucial for introducing new programs. The 
communication educates parents about what is 
happening in schools and it can garner support 
for innovation and change. Lack of communi- 
cation can create a barrier: it may result in 
parents working to maintain the familiar status 
quo in opposition to the newly mandated 
school performance program, or not becoming 
fully involved in maximizing and coorainating 
home and school educative efforts. A building 
principal told us: 



Many times it has to do with a concern for 
what a parent’s or a group of parents’ 
reactions might be. So then we might need 
to take a look at what we need to do 
with communicating better with parents . 
This way parents can better understand 
the changes of instruction and our instruc- 
tional purposes. (RF.0405) 



A state-level a;.sessment and curriculum 
reform initiative caimot but benefit from related 
teacher initiative. Teachers shoulder much of 
the burden of change in a mandated high-stakes 
program, and the quality and speed of transi- 
tion from one program to another can be 
enhanced by clear communication. Unfortu- 
nately, clear communication about the intent 
and nature of the school pet formance program 
was not apparent to some of the participants in 
the study. A fifth-grade teacher reported: 



We knew that we were going to use litera- 
ture, but we were going to be using it 
under a different structure. We were going 
to be using it under something called the 
“Formula 3 Program” and then kind of 
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like mysteriously as we got to know each 
other and as the school year began, we 
started to hear different language. And 
people were starting to say different things 
to us. Things we hadn’t heard when we 
were interviewing to come here. The 
pieces were given to us little by little in 
pieces. So it has taken us up to this point 
to start to really see what the whole thing 
is supposed to look like. (RF.0402) 

This fifth-grade teacher reported that she 
received considerable praise and encourage- 
ment from her principal for changing instruc- 
tion, while doubting her own ability. This 
concern was compounded by the fact that when 
she asked district personnel for an instructional 
model that was based on and aligned with the 
statewide performance assessment, she was told 
that she was the model. The possible seren- 
dipity of the situation was not comforting to the 
teacher. The following excerpt demonstrates 
the barrier that was caused by lack of detailed 
orientation to the mandated program, and what 
the teacher perceived to be haphazard direction 
and guidance once the program was in place. It 
was not clear if the state, district, or both 
should have been working to provide teachers 
with models of what is meant by good practice. 

There is no unit or model of unit develop- 
ment. It’s “Hey, don’t worry about it. 
Take risks. The kids will do great because 
you’re great.” I, at various meetings with 
the folks who are telling us to take risks 
and don’t worry have said, “Give me 
some help. Show me how to do it.” lam 
wonderful, but if you say it can be done, 
then show me how. What does it look 



like? The response is that we are the 
model and that this school is the model 
and that this is how it should look. It is 
pretty scary when they say we are the 
model at this school. It is flattering, but it 
is scary because we feel like we need 
more guidance, and the people we are 
looking to for guidance are looking to us 
for guidance. (RF.0402) 



The importance of communication in 
facilitating the change needed to implement the 
school performance program was a thread that 
ran through many participants’ comments and 
responses to our question about barriers. As a 
follow-up, we contacted state education person- 
nel to check on the availability of information 
that administrators, teachers, and students 
might use to learn more about the assessment 
program. We were told that the state had no 
standardized method for disseminating infor- 
mation to teachers about me purpose or goals 
of the school performance progran.;. There was 
no formal documentation intended to familiar- 
ize teachers with the performance assessment, 
or to provide detailed explanations of perfor- 
mance assessment ways and means. We found 
that the state did send annotated scoring 
guides and sample tasks and student responses 
to each district. We also found that a booklet, 
“Teacher to teacher talk” (containing actual 
quotes from teachers about the school perfor- 
mance program) was sent to districts. It was 
expected that the booklet would reach teachers , 
but there was no suggested or mandated routine 
or program of dissemination. None of the 
teacher participants reported familiarity with 
the booklet or its contents. 
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Conclusion 

The results of this study demonstrate that 
the successful implementation of a school 
performance program devised to change curric- 
ulum and instruction and assessment requires 
more than the development of assessment 
materials and procedures. It requires attention 
to (and the involvement of) the people who 
administer, take, and use assessments. Despite 
the fact that participants worked in schools that 
were perceived by district representatives as 
successful in their implementation of MSPAP 
related curriculum and instruction, the partici- 
pants cited five general classes of barrier. It 
may be the case that barriers are more pro- 
nounced and more numerous in schools that 
are not successful in meeting the changes 
demanded by MSPAP. There also may be 
additional barriers that were not cited by the 
people we worked with. 

A first barrier was the lack of alignment of 
current classroom practice in instruction and 
assessment with the mandated performance 
program. This is not surprising, given that the 
school performance program was intended to 
change school practice— in some cases to 
change it drastically. The participants in this 
study made clear the fact that change takes 
time, and change as massive as redesigning 
curriculum in accordance with a performance 
program requires considerable efforts and 
expenditures. School personnel reported 
examples of lack of alignments that required 
attention to the planning and carrying out of 
instruction, teachers’ interactions with students, 
and roles and interactions of teachers and 
administrators. 



A second barrier was the lack of alignment 
in the beliefs and values that underlie teachers ’ 
instruction and assessment. We did not 
encounter frequent reports of teacher resis- 
tance to change as a barrier. However, teacher 
supervisors cited teacher resistance as a poten- 
tial barrier. We note again that the district 
and school selection procedures in this study 
produced a sample of schools in which the 
implementation of change was going well, 
according to school administrators. None of the 
schools appeared to have personnel strongly 
opposed to the change encouraged by the state 
program. We are not sure that this would have 
been the case in schoc’s in which lack of 
alignment between the performance program 
and teacher belief and practice was pronounced. 

Insufficient resources for meeting the 
demands of the performance program was a 
third barrier cited by many participants. 
Money (in the form of financing and supporting 
teacher inservice work sessions, instructional 
innovation, and teacher collaborative work 
groups) appears to hold considerable promise 
for making efficient the evolution of instruction 
and learning tied to a mandated performance 
program and assessment. It is probable that the 
availability of time and money will continue as 
a central factor in programmatic change related 
to performance programs. 

A fourth barrier to the implementation of 
the performance program was the performance 
assessment materials and procedures them- 
selves. The performance assessment included 
test security measures that prevented teachers 
from preparing adequately for the administra- 
tion of the assessment. In addition, assessment 
policy required that all students take the 
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assessment, whether or not there was clear 
indication that the student could succeed. 
Teachers reported concern that students’ public 
failure might compromise gains in achievement 
attributable to the performance program. The 
assessment routine also required that students 
be mixed with unfamiliar students and take the 
assessment in unfamiliar surroundings, and 
school personnel saw this as a potential barrier 
because of the heightened anxiety it might 
cause in students. Other barriers related to the 
assessment itself were the timeliness of report- 
ing of school performance program results, and 
the lack of usefulness of the information that 
the ^sessment yielded. 

The barriers related to communication 
were considerable, especially with regard to 
the connections between the performance 
assessment and classroom curriculum and 
instruction. Important information about 
performance program details was to “trickle 
down” (as worded by a state education official 
central to the school performance program) 
from the state education department to school 
districts to schools to teachers in classrooms. 
Yet there were indications that much important 
information was not getting to the classroom 
teachers who sought it and required it. School 
personnel were placed in the position of having 
to use the performance assessment without a 
clear understanding of its purpose and nature. 
In addition, participants cited instances in 
which contradictory and incomplete informa- 
tion was communicated to districts and 
teachers. Given the intended consequences of 
the school performance program, the develop- 
ment of a communication system that carries 
clear and pertinent information to all those who 



have a stake in the assessment should be a 
priority. Otherwise, curricular and assessment 
practice and change may be based on a chain of 
inferences that is tenuous and inaccurate. 

The variety of barriers confronted by 
school personnel and students demonstrates 
that the relative ease of mandating large-scale 
systemic change is in sharp contrast to the 
process of realizing that change. In this study, 
the mandate was strong and the stakes were 
high, but the means for moving districts, 
schools, classrooms, teachers, and students 
toward academic achievement and performance 
assessment goals were not always apparent or 
present. One interpretation of this situation is 
that the state (having mandated a generic type 
of change) encourages local control of the 
details of change. We did not detect a commu- 
nication system that could support such an 
arrangement. We are concerned with the 
number of participants who reported difficulty 
in understanding important aspects of the 
rationale and rature of the school performance 
program. 

The Maryland School Performance Pro- 
gram is intended to drive innovation and 
change in curriculum and assessment. The 
successful joining of the initiatives and goals is 
accompanied by barriers that may undermine 
the intended effects of statewide mandates, 
and that may prevent schools, teachers, and 
students from benefiting from these changes. 
The driver of change in the Maryland School 
Performance program is performance assess- 
ment, considered the key to curricular reform 
when connected with effective instruction 
and learning. In addition, the mandated per- 
formance assessments are considered central to 
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efforts that help students learn and meet 
existing and evolving educational standards. 
Recently, educators, politicians, and researchers 
have noted the importance of assuring school 
delivery standards so that children from dif- 
ferent circumstances may have equivalent 
opportunities to learn and meet educational 
standards. We believe the results of this study 
signal at least one area that needs similar 
attention: there should be standards for man- 
dating school change, and for developing and 
using large-scale performance assessment. 
These standards will help anticipate barriers to 
the implementation of programs intended to 
change instruction, and the standards may 
assure that issues of curricular alignment, 
school resources , communication , involvement , 
and professionalism of all people involved in 
the effort are given a priori and ongoing 
attention. 
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